|
KMID : 1143320240130010065
|
|
Therapeutic Science for Neurorehabilitation
2024 Volume.13 No. 1 p.65 ~ p.74
|
|
|
Republic of Korea and the National Research Foundation of Korea
|
|
Hong Jun-Hwa
Kim Na-Yeon
Min Hye-Min
Yang Ha-Min
Lee Si-Hyun
Choi Seo-Jin
Park Jin-Hyuck
|
|
|
Abstract
|
|
|
Objective : This study assessed ChatGPT, an artificial intelligence system based on a large language model, for its ability to pass the National Korean Occupational Therapy Licensure Examination (NKOTLE).
Methods : Using NKOTLE questions from 2018 to 2022, provided by the Korea Health and Medical Personnel Examination Institute, this study employed English prompts to determine the accuracy of ChatGPT in providing correct answers. Two researchers independently conducted the entire process, and the average accuracy of both researchers was used to determine whether ChatGPT passed over the 5-year period. The degree of agreement between ChatGPT answers of the two researchers was assessed.
Results : ChatGPT passed the 2020 examination but failed to pass the other 4 years’ examination. Specifically, its accuracy in questions related to medical regulations ranged from 25% to 57%, whereas its accuracy in other questions exceeded 60%. ChatGPT exhibited a strong agreement between researchers, except for medical regulation questions, and this agreement was significantly correlated with accuracy.
Conclusion : There are still limitations to the application of ChatGPT to answer questions influenced by language or culture. Future studies should explore its potential as an educational tool for students majoring in occupational therapy through optimized prompts and continuous learning from the data.
|
|
|
KEYWORD
|
|
|
Artificial intelligence, Large language model, Occupational therapy licensure examination
|
|
|
FullTexts / Linksout information
|
|
|
|
|
|
Listed journal information
|
|
|
|